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ABSTRACT 


This  report  presents  a non^technical  account  of  a research  program  detigxked 
to  investigate  personnel  problems  associated  with  electronic  maintenance  and  operation. 

The  procedures  employed  for  obtaining  objective  descriptions  aboard  ships  op- 
erating in  the  Pacific  area  and  a description  of  groups  obsert'ed  are  given. 

Rationale  underlying  the  development  of  a battery  of  observational  techniques  is 
presented,  and  each  of  the  techniques  is  described  in  general  terms. 

A brief  discussion  of  the  treatment  of  the  data  is  given,  and  evaluations  of  the 
methods  are  presented. 

The  paper  is  concluded  with  some  general  evaluative  comments  concerning  the 
use  of  a multi -method  approach. 
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A DESCRIPTION  OF  THE  RESEARCH, 


INTRODUCTION 


The  nuTuose  of  this  repoi*t  is  to  present  a non-technicai  account  of  a program  of 
research  designed  for  making  preliminary  observations  of  the  shipboard  electronic  situ- 
ation with  an  emphasis  on  the  maintenance  aspects  of  the  situation.  The  observational 
techniques  used  and  the  methodological  problems  Involved  are  discussed.  More  techni- 
,tE  Cif  this  sric  lr>v»rtlwatto?*  are  contained  in  other 


reports  of  this  series. 


STATEMENT  OF  THE  PROBLEM  AREA 


The  primary  objective  of  the  investigation  was  to  obtain  an  accurate  description 
of  shipboard  electronics.  Such  a description  is  essential  to  the  eventual  organisation  of 
the  research  area  and  it  is  prerequisite  to  the  formulation  of  specific  research  hypothe- 
ses. In  addition,  the  examination  of  such  a broad  area  d'oring  the  initial  stages  of  a re- 
search effort  yields  important  by-products.  For  one  thing,  the  information  derived  is 
of  immediate  interest  in  its  own  right.  While  it  does  not  provide  answers  to  rigorous 
experimental  questions,  such  a description  of  the  shipboard  electronic  maintenance  situ- 
ation Is  of  general  value  to  Ihose  charged  with  the  responsibility  of  providing  trained 
men.  A second  important  by-product  e£  such  research  is  the  opportunity  it  provide*  for 
the  try  out  of  new  research  methods  which  may  be  adapted  to  later  experimental  efforts. 

During  the  planning  stages  of  this  study  a number  of  alternative  approaches  to 
the  problem  of  developing  detailed  and  accurate  descriptions  were  considered.  Perti- 
nent written  material  was  examined  and  evaluated  as  a source  for  describing  the  elec- 
tronic maintenance  activities  performed  by  members  of  ships*  crews.  These  sources 
did  not  provide  the  detailed  description  which  was  desired  for  this  research.  It  was  de- 
cided that  the  problem  of  fleet  description  could  best  be  accomplished  by  more  direct 
meeins. 

Field  work  as  aui  initial  stage  of  a research  program  has  the  advantage  of  pro- 
viding an  opportunity  for  research  personnel  to  live  y/ith  the  problems  with  which  they 
are  to  work.  Productive  research  insights  are  more  likely  to  occur  in  this  setting.  In- 
timate acquaintance  with  the  problem  area  is  particularly  valuable  since  the  eventual 
decision  as  to  the  feasibility  of  shipboard  research  of  various  kinds  cache  determined 
only  by  those  who  have  detailed  knowledge  of  both  the  shipboard  situation  and  the  re- 
search potential  of  the  group. 

The  decision  to  do  field  research  was  essentially  one  as  to  where  the  research 
was  to  be  done.  Once  this  judgment  had  been  made,  ways  for  accomplishing  the  research 
were  considered.  Three  alternative  approaches  were  examined  and  evaluated.  They 
were  unstructured  observation,  the  application  of  a single  observational  technique,  and 
the  concurrent  use  of  several  different  observational  techniques. 

Unstructured  observation  coiisists  of  observing  without  plan  or  limitation.  The 


^.iriawipau.  e«vi9i»va.g«!  £«  »«cB  oDscrmtiQn  IS  ta»i  s*  <sces  iRot  cssTTimit  the  ohaerwr  to  s ti0.d 
gclwsdiile  or  ».  fixed  procedisT**,  He  is  left  free  to  oh*er<<«!,  directing  Ws  ofeseryiitton  as 
ha  sue#  fit.  During  World  War  II  the  preliminary  steps  o£  Opcratioas  Aisaiysia  e^’ten  todE 
the  form  of  trained  observers  "J«*i  looMag'"  with  the  objective  of  discovering  aod  gain- 
ing irusight  Into  the  proble-na  of  arj  area  ?a  which  they  were  reltstively  naive,  Qttiie  often 
snch.  an  observer  would  see  a problem  in  a isew  way  and  in  certain  cases  could  SE^gesi 
Ingenious  solutions  to  it.  The  succeaa  of  such  proceduns*  is  probably  due  to  fee  de- 
tached objecti  vity  oi'  fee  obs«r%‘^cr»,  their  freedom  to  observe,  and  their  ability  to  trams- 
fei'  %aeas  ana  rnethccls  trom  old  settings  to  new  ones.  The  principal  difliciilly  wife  the 
procedure  is  that  there  ic.  no  ouarantee  that  an  *'on<»Ti-»yerf  '»ppr"*cl! 

will  be  either  efficient  or  effective. 


The  application  of  a single  obserr^ationai  technique  consists  of  applying  a sele 
{cr*  c vc'ry' 'cvr*a,l  iwuxxhcr'  w,  excAijcf^iveAy,  xhia  ap- 

proach is  particxUarly  suited  to  those  well-structured  situations  where  problems  have 
been  carefully  defined  and  reduced  to  testable  form.  Under  these  conditions  it  is  most 
efficient  and  economical  to  devise  a specific  method  for  each  specific  problem.  This 
approach,  however,  has  an  inherent  difficulty.  It  is  ao  selective  of  information  that  much 
of  fee  data  which  would  be  pertinent  to  the  broad  understanding,  appreciation,  and  organ- 
isation of  a domain  is  not  collected. 


The  concurrent  use  of  several  different  observational  techniques  is  essentially 
an  eclectic  approach.  It  employs  a battery  of  ooaervationai  methods  which  complement 
one  another  in  such  a manner  as  to  comprehensively  cover  the  total  domain  being  ob- 
served. The  advantages  of  each  method  are  capitalized  upon  while  the  weak  points  of 
each  are  overcome  by  fee  use  of  other  methods. 

The  consideration  of  fee  three  alternative  approaches  (namely;  unstructured  ob- 
servation, fee  application  of  a single  observational  technique,  and  the  concurr-,jnt  u^e  of 
several  different  techniques)  led  to  the  decision  to  adopt  the  third  alternative.  This  ap- 
proach came  to  be  known  as  the  multi -method  approach.  It  will  be  referred  to  in  this 
manner  throughout  the  series. 


THE  MULTI-METHOD  APPROACH 


A number  of  specific  objectives  for  fee  multi-method  approach  were  formulated. 
Many  kinds  of  information  were  desired  from  various  sources.  The  major  effort  was  to 
obtain  objective  and  unbiased  descriptions  of  the  Jobs  that  electronic  personnel  perform. 
These  descriptions  were  sought  from  three  different  sources;  members  of  the  resesTreh 
group  who  observed  and  reported  fee  behavior  of  the  men  on  the  job:  the  men  them- 
selves; and  fee  supervisors  of  the  men.  Each  of  fee  sources  offered  specific  but  differ- 
ing advantages.  The  man's  own  account  of  his  job  could  be  quite  detailed  but  it  could  fee 
distorted  by  such  factors  as  the  personality  of  the  man  and  His  lack  of  objectivity.  Too 
observers'  account  of  the  man's  activity  was*  more  objective  but  limited  by  time  and 
situational  factors.  The  supervisors’  descriptions  were  subject  to  still  another  set  €s£ 
biases.  Since  each  source  of  dcsscriptlon  presented  a slightly  different  picture,  a com- 
posite of  fee  three  was  used  to  uncover  the  many  different  facets  of  the  situation.  Tech- 
niques were  introduced  for  the  purpose  of  obtaining  descriptions  of  fee  men  who  made  up 
fee  sample.  These  techniques  sought  relevant  £fxts  concerning  the  present  atatm  of  ih® 
men,  their  past  history  (e.g,,  time  In  the  service,  past  assignments,  and  previous  traisi- 
Ing),  and  their  future  plan.  . Descriptions  of  the  jobs  and  descriptions  of  the  man  were 
c<3mpiemented  by  deecrlptlon.®  of  the  physical  plant,  available  tools,  test  equipment,  stow- 
age facilities,  fend  records. 
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In  addition  to  variot;itB  descriptions , the  battery  Intluded  methods  -addcb  required 
certain  judgments.  Electronic  maintenance  personnel  Judged  ttie  relevance  of  certain 
training  topics  and  exercises  to  the  maintenance  job.  These  men  also  judged  the 

degree  tsf  skill  and  the  amount  of  electronics  comprehension  required  to  do  a certain 
task.  The  officers  who  were  In  charge  of  the  different  group*  o£  electronics  personnel 
judged  the  imporiance  of  a number  cf  specific  abilitle*  to  the  proficiency  with  which  the 
job  was  dons.  They  also  ranked  the  abilltie*  from  **most  important  to  the  job"  through 

*lea5>-|  imrertrtsrit  *• 

- t#  ' 

»<ey  vs?ere  also  asked  to  give  their  opinions  regarding  Navy 

problems,  Ihese  opinions  took  the  form  of  selecting  which  of  several  available  solutions 
was  "best.”  No  attempt  was  made  to  determine  the  attitudes  of  the  respondents  oi  to 
assess  their  morale.  It  is  important  to  keep  in  mind  that  this  Investigation  wa«  easen- 
ttaiiy  non-evaluative  throughout. 

The  multi-method  battery  covered  one  other  area.  Detailed  accounts  of  specific 
past  event®  were  requested  from  some  members  of  the  sample  for  the  purpose  of  ex- 
tending the  range  o£  the  observations  beyond  the  temporal  limitations  imposed  by  the 
fact  tiiat  the  observers  were  aboard  each  ship  for  a weeh  or  less. 


DEVELOPMENT  OF  INDIVIDUAL.  METHODS 


The  selection  of  methods  to  accomplish  the  objectives  stated  above  was  based 
upon  a number  of  considerations.  A method  would  be  included  only  If  it  promised  to  con- 
tribute relevant  information.  Because  of  the  descriptive  nature  of  the  approach,  the 
limits  of  relevancy  were  broad.  Items  were  included  if  the  information  was  suspected  to 
be  relevant.  The  relationship  between  the  methods  was  also  considered.  Methods  which 
yUIded  duplicate  uiformaiion  were  eUminated  tmless  part  of  their  purpose  was  to  pro- 
vide a check  upon  the  other  methods.  The  more  objective  and  reliable  methods  were 
selected.  Proposed  plans  of  analysis  also  governed  the  selection  of  the  methods.  Those 
methods  which  permitted  the  most  straightforward  analytical  treatment  were  used  when- 
ever possible.  In  addition  to  these  rather  general  considerations,  there  were  certain 
criteria  which  served  as  limits  in  the  choice  of  observational  techniques.^  A method 
could  not  be  used  unless  it  was  sufficiently  portable  to  be  carried  aboard  ship.  Care 
was  taken  that  the  methods  could  be  administered  by  one  man  within  the  situational  liznt- 
tationa  of  a destroyer  at  sea.  Time  restrictions  were  taken  into  consideration,  and  each 
method  had  to  be  accomplished  without  Interfering  with  the  activities  of  shipboard  per- 
sonnel. 


A n'umbor  of  different  sources  were  used  in  the  development  of  the  subject  s»at- 
ter  of  the  various  method®.  Navy  manuals,  textbooks,  and  forms  were  of  great  assist- 
ance. Civilian  textbooks  in  the  areas  of  electronics  and  electricity  provided  many  une- 
fsd  items.  Research  conducted  for  other  branches  of  the  armed  services  and  for  indvLS- 
iry  contributed  to  «ievsr&i  of  tlte  methods.  Much  of  the  subject  matter  and  a great  deal 
of  practical  advice  was  obtained  from  conferences,  interviews,  and  conversations  with 
various  Naval  personnel. 

Proposed  observational  methods  were  subjected  to  try  outs  of  various  sort*.  The 
principal  effort  in  this  direction  was  to  try  ou‘‘  the  method  on  board  a destroyer  that  was 
undergoing  a shakedown  period.  At  the  concluoiion  of  each  trip  the  method  was  evaluated 
in  terms  of;  the  &rm  of  analynts  thnt  was  proposed  for  it;  the  relationship  of  the  method 
in  question  to  tlte  other  methods;  administrative  restrictions  that  were  encountered  dur- 
ing the  try  out  trip;  the  reaction  of  the  men;  effort  requirements;  situational  limitations; 
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sa^iipie  yecdffding  prcblenis  encountered;  time  limitations;  and  potential  sources  o£ 

Mas,  Appropriate  revisions  were  made  and  the  method  wajs  taken  to  sea  again.  Methods 
were  subjected  to  try  out  in  the  order  in  which  they  were  completed  with  the  result  that 
some  of  the  methods  received  more  extensive  pretesting  than  others. 


In  addition  to  the  shipboard  personnel,  two  other  groups  were  employed  from 

tlSTS#  to  timO  to  rc»*oij»i«t  0(« 


of  UWATE  acted  as  subjects  and  offered  helpful  criticism  erf  some  of  the  methods.  Dur- 


il»«  vei-y  tf»rly  siagea  of  the  development  oi  some  o£  the  methods  coUege  students 
who  had  served  ns  electronics  technicians  during  World  War  n served  as  subjects. 


THE  OBSERVATIONAL  PROGRAM 


Electronics  personnel  abc&rd  twenty  ships  of  the  destroyer  class  made  up  the 
final  observational  sample.  One  of  the  reasons  ^at  ships  of  this  class  were  chosen  was 
that  they  were  less  specialise  as  to  duty  than  many  of  the  ships  that  were  smaller  or 
larger.  This  lack  of  specialisation  offered  the  maximum  possibility  of  generalization  of 
the  data  to  ships  of  other  types.  Destroyers  have  every  major  class  of  electronic  equip- 
ment, and  they  encounter  most  of  the  electronic  problems  that  any  class  of  ship  might  be 
expected  to  encoimter.  Another  reason  for  selecting  destroyers  as  the  class  of  ship 
upon  which  i^e  observations  were  made  was  the  fact  that  a stifficient  number  of  different 
ships  of  this  type  are  available  to  permit  the  collection  of  the  data  within  practical  time 
limitations. 

Administrative  arrangements  for  the  observational  tours  were  made  in  coopera- 
tion with  the  office  of  the  Commander  of  the  Cruiser-Destroyer  Force  of  the  Pacific 
Fleet.  The  ships  were  notified  of  the  observers'  arrival  about  one  day  in  advance. 

The  observations  took  place  while  the  ships  were  engaged  in  training  operations 
at  sea.  These  operations  were  of  the  usual  type  and  included  almost  all  of  the  different 
destroyer  missions  with  the  exception  of  shore  bombardment. 

Each  observational  team  was  composed  of  two  members.  The  decision  to  use  a 
team  of  this  size  was  a result  of  the  try  out  trips  which  revealed  that  a single  observer 
wotild  not  be  able  to  accomplish  the  observational  mission  within  the  allotted  time  and 
that  shipboard  facilities  were  too  limited  to  accomodate  larger  groups. 

Trips  varied  in  length  from  2|  days  to  days  with  an  average  of  about  four  days 
per  trip.  Usually  the  ship*  steamed  at  night,  but  in  some  cases  they  anchored  or  re- 
turned to  harbor.  On  a typical  trip  the  observers  would  board  the  vessel  on  Sunday  even- 
ing, and  the  ship  would  get  under  way  early  on  Monday  morning.  Throughout  the  trip  the 
observers  worked  independently  except  that  tliey  coordinated  their  activities  so  as  to 
prevent  duplication  of  effort  and  unintentional  omission  of  data.  The  early  portion  of  the 
trip  was  spent  acquainting  the  officers  with  the  purpose  of  the  observations  and  enlisting 
their  cooperation.  The  remaining  time  was  spent  participating  In  the  ship's  exercises 
as  an  observer. 

Efforts  vwre  made  to  ensure  standard  observational  procedures  during  the  course 
of  the  study.  The  observers  received  training  on  how  to  employ  the  various  methods. 
This  training  period  included  orientation  trips  and  sea-going  try  outs  with  the  methods. 
Another  means  of  maintaining  standard  procedures  was  to  rotate  the  observers  so  that 
no  two  observers  consistently  worked  together.  This  process  of  rotation  allowed  the 
project  director  to  ride  a®  an  observer  v/ith  each  of  the  other  observers  and  ihxis  dia- 
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cover  and  correct  any  deviation  from  standard  practice.  A third  means  of  minimizinf 
the  variance  between  observers  was  the  use  of  an  observer’s  manml.  This  manuai  gave 
•cookbook**  instructions  as  to  exactly  how  each  of  the  methods  should  be  used. 

Although  it  was  desirable  to  cover  the  entire  area  of  electronics,  it  became  ap- 
parent quite  early  in  the  planning  of  the  studv  that  certain  concesf»ir»Ttfl  ihn.^  +«  v.®  ^.*>*5* 
reduce  the  domain  to  a workable  size.  One  of  the  most  important  of  these  was  to  give  up 
the  idea  of  maVmo  s '•omplete  end  comprehensive  description  of  Vslu  lli«  uporaiing  and 
maintenance  aspects  of  electronics  activities.  The  attention  of  this  study  came  to  be 
focused  on  maintenance.  The  maintenance  data  are,  therefore,  more  detailed  and  volum- 
inous than  the  non-maintenance  data. 

The  sample  of  shipboard  electronics  personnel  was  further  restricted  because  of 
other  considerations.  Some  methods  cotild  not  be  used  as  widely  as  they  might  have  been 
due  to  their  time  consuming  nature.  This  was  particularly  true  of  the  interview  which 
might  have  been  used  more  extensively  if  each  had  not  required  approximately  an  hour 
of  the  observer’s  time.  The  same  time  limitation  led  to  the  restriction  of  the  card  sort 
sample.  Each  sorting  took  so  long  that  it  was  not  feasible  to  have  all  of  the  radarmen 
and  the  radiomen  participate.  As  a resul'l^  only  the  leading  petty  officer  for  each  of  these 
ratings  attempted  the  card  sort. 

Opinions  were  sought  from  those  members  of  the  sample  who  were  in  such  a po- 
sition that  the  particular  opinion  held  could  influence  policy.  For  this  reason,  most  of 
the  sample  for  the  general  questionnaire  was  made  up  of  officers.  The  samples  for  cer- 
tain other  techniques  were  limited  to  persons  in  supervisory  positions.  A detailed  siim- 
mary  of  the  sub-samples  to  whom  the  various  methods  were  administered  is  included  at 
the  end  of  the  next  section  of  this  report. 


DESCRIPTION  OF  METHODS 


A form  of  diary  keeping  was  introduced  into  the  methodology  as  a means  of  re- 
cording direct  observations  of  the  men,  their  jobs,  and  the  gear.  The  diary  was  essen- 
tially a technique  in  which  the  observer  recorded  in  a time  sequence  all  that  took  place 
vrithin  a predetermined  area  of  observation.  Observations  were  recorded  either  in  writ- 
ing or  on  a tape  recorder.  The  diary  accounts  were  oriented  about  the  man-on-the-job, 
the  electronic  equipment,  and  the  trouble  shooting  sitvation.  The  man-oriented  diaries 
required  that  the  observer  accompany  a msn  and  observe  everything  that  he  did  in  a 
given  period  of  time.  For  the  place-oriented  approach  the  observer  took  a pcsiticn  la  an 
area  where  electronic  equipment  was  installed  and  recorded  the  activities  of  the  men  ia 
that  area  during  a specified  time  interval.  Examples  of  such  "places*'  are  CIC,  radio 
central,  transmitter  room,  etc.  Emphesis  was  placed  on  trouble -oriented  diaries.  When- 
ever word  was  received  of  an  electronic  failure,  an  observer  would  go  to  the  scene  of  the 
repair  and  record  in  detail  an  acectxnt  of  the  trouble  shooting  activity  until  the  equipment 
was  again  operating. 

Another  method,  a check  list,  vras  used  for  the  recording  of  direct  observations 
by  the  observer  and  for  securing  descriptions  by  supervisors  of  the  work  done  by  the 
men  under  their  supervision.  Statemesits  for  the  check  list  were  chosen  to  match  the 
generality  level  of  standard  Navy  job  description  materials  used  in  the  fleet,  such  as  the 
Manual  of  Qualifications  for  Advancement  in  Rating.  The  job  statements  were  checked 
according  to  the  rate  of  the  man  observed  and  the  class  of  equipment  involved. 

With  regard  to  the  msnipulatton  of  various  electronic  equipment,  terms  such  as 
“caltbratos,*  “operates,*  “measures  eencitivity  of,"  “dees  preventive  maintenance  upon" 
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*iwi  "trains  personal  'pttforming  corrective  msiofeenance  on,"  «ie  typical  ol  tibe  list. 
Further  emphasis  was  placed  on  the  use  of  test  equipment.  Items  not  related  to  specific 
equipment  ware  also  Included,  x^xaznples  of  these  are  “orders  replacemeat  parts,** 
“maintains  records  of  electronic  equipment,**  and  “refers  to  circuit  wiring  diagrasna.** 


When  the  check  list  wee  used  for  direct  oheerVetiomj  the  oh«erw*r  ehecVed  the 


appropriate  atatements  while  watching  the  worker  perform  a given  task.  As  an  indi* 

rcct  ww££XVu.tlOi«^l  cIa4^CK  XI«tl  nileu  oul  iXk«£  pvliy  0Jui.4:ejr« 

He  used  his  knowledge  c#  the  activities  of  his  men  as  » basis  for  his  checks.  Enlisted 


supervisors  were  used  for  radarmen,  radiomen,  sonarmen,  electronics  technicians,  and 
fire  control  technicians.  The  only  officer  called  upon  to  complete  a check  list  was  the 
electronics  repair  officer. 


In  an  effort  to  gat  judgments  of  the  imports-nce  of  certain  abilities  to  given  Jobs,  a 
rating  scale  was  constructed  for  use  by  supervisors.  It  was  called  an  ability  reqtnre- 
ments  scale  and  its  use  was  intended  to  be  exploratory.  The  scale  consists  oi  nineteen 
items  which  endeavored  to  describe  independent  and  non-overlapping  ability  traits.  The 
items  were  designed  to  capture  in  a single  statement  the  central  meaning  of  a trait  which 
had  previously  been  defined  by  factor  analysis.  Examples  of  these  trait  desl&nationa  are 
•sbility  to  do  arithmetic  problems,  such  as  adding,  subtracting,  multlpiying,  and  divid- 
ing"; "ability  to  put  practical  problems  into  metbematical  terms  so  that  they  can  be 
solved";  "ability  to  lay  plans  and  foresee  problems":  and  "ability  to  solve  problems  in- 
volving mechanical  things  such  as  gears,  levers,  etc." 


Each  trait  was  evaluated  in  two  wa^is;  (1)  on  a five-point  scale  indicating  the  rel- 
ative amount  of  the  particular  ability  required  for  the  job,  and  (2)  by  a rank  ordering  of 
all  traits.  The  job  of  the  electronics  technician  was  evaluated  by  the  lead  ET  and  the 
electronics  officers,  the  radarman*s  job  by  the  CIC  officer,  the  sonarman’s  billet  by  the 
ASW  officer,  the  radioman’s  job  by  the  comriiunications  offices  and  the  work  of  the  fire 
control  technician  by  the  assistant  gunnery  officer  in  charge  of  fire  control  equipment. 

A job  questionnaire  was  included  among  the  methods  to  take  advantage  of  the 
simple  and  economical  means  for  obtain'ng  job  information  which  the  questionnaire  ap- 
proach provides » In  addition  to  dercriptions  of  pertinent  duties,  it  contained  questions 
covering  materials  used  on  the  job  and  items  relating  to  the  personal  history  of  tl.e  man 
questioned. 

Examples  of  the  biographical  data  items  are: 

"How  long  have  you  been  on  this  ship?  " 

“What  job  titles  did  you  hold  before  being  assigned  your  present 
job  title?" 

"List  the  xtemes  and  locations  of  Navy  schools  you  have  attended 
and  indicate  the  time  spent  in  each." 

Job  description  Items  ranged  from: 

“Check  as  many  of  tlte  following  publications  as  you  use  in  your 
work  on  this  ship.” 

“Check  below  each  of  the  pieces  of  test  equipment  you  have  used 
in  your  work  on  this  ship.” 


to. 
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if  least  five  disties  which  you  perform  when  at  your 

"battle  siaticn  and  after  each  of  these  duties  estimate, the  percentage  of 
time  yon  spend  while  ai  hattle  gfeatiots  doii^  this  duty.  The  sum  of  ail 
of  these  percentages  should  equal  WO  per  dent," 

These  ^questionnaires  were  completed  by  all  enlisted  electronics  personnel  on  the 
■shtpc.  The  ferrS'^o  Tuulto  by  ttuperviaors  and  couected  by  them  wnen 

completed. 

Officers,  through  an  eiqrression  of  their  opinions  on  crucial  matters,  can  contrib- 
ute much  tfUfonriaticn  regarding  the  relative  irnporiance  of  certain  personnel  factors.  To 
get  the  reactiona  of  officers  in  the  area,  cf  electronics,  a general  questionnaire  was  in- 
trcvduced.  ~ 

Opiiuojss  were  sought  regarding  the  classification  and  utiUsation  of  personnel. 
F or  ejcampie,  on*  Item  asked  what  per  cent  of  the  preventive  maintenance  work  on  radars 
should  be  done  by  radarmen  and  what  percent  by  electronics  technicians.  Another  ques- 
tion asked  what  tite  electronics  technicians  should  do  when  all  of  the  gear  was  v/orking. 
Still  other  questions  related  to  matters  of  training*  use  of  publicatioas  and  I'ecords,  and 
causes  of  excessive  shut-down  time,  la  all  cases*  items  were  written  to  receive  defini- 
tive answers.  They  'were  predominately  of  the  yea  - no,  multiple  choice,  or  ranked  al- 
ternative type. 

In  general  the  judgments  were  made  bytbe  officer  abo#|  the  sUuatlofe  faced  by 
the  men  ujcrfer  him.  That  is,  the  CIC  officer  esq^reftsed  his  opimoiqgphiefly  with  reference 
to  the  CIC  situation,  the  ASW  officer  with  reference  to  sonar,  the  gunnery  officer  with 
reference  to  fire  control,  etc.  In  thia  way,  CIC,  ASW,  electronics  repair,  commimira- 
tlons,  gunnery,  englueeriug,  «oud  operations  personnel  were  .represented. 

The  critical  Incident  technique  was  employed  to  get  anecdotal  data  on  several 
questions  for  which  this  tecimique  tsMrtlcula7;ly  adapted.  Some  of  these  questions  con- 
cerned: personnel  factors  which  contributed  td^equlpment  malfunction,  causes  of  exces- 
sive shut-down  time,  limitations  placed  upon  maintenance  activities  by  equipment  design, 
and  critical  job  requirements  for  electronics  personnel.  Other  questions  sought  de- 
scriptions of  incidents  which  were  '‘typical**  of  unusually  successful  malntewance  be- 
havior and  situations  where  electronics  technicians  were  instrumental  in  the  repair  of 
sonar  and  fire  control  equipment. 

Seven  questions  were  used,  each  struetwed  along  the  lines  of  one  of  the  issues 
abarve.  These  were  asked  la  a private  interview  situation  writli  6»C  h electronics  techni- 
cian. A typical  question  was,  "There  are  occasions  when  the  repair  of  electronics  gear 
is  not  accomplished  as  quickly  «s  it  might  be.  Clan  you  tell  me  about  a particular  inci- 
dent where  a piece  of  gcai’  was  shut  down  for  a longer  time  than  it  should  have  been?** 
The  entire  interview  was  tape  recorded,  and  the  incidents  were  later  transcribed  for 
analysis. 

The  iyalhthg  'qtiefcttoftnialrtg  *WM  ddVblophd  to  collect  the  judgments  of  electronics 
teChnlClSttS  nl  Sbleeted  training  curriculum  toptcc  to  their  mainte- 

^nance  job.  This  consisted  of  a list  of  211  standard  curriculum  topics,  each  of  which  the 
msn  on  the  job  evaluated  as  to  its  usefuiness  in  his  assigned  job.  Topics  to  be  rated 
were  selectedl  from  the  subject  matters  m basic  electricity  and  electronics,  and  the 
principles  of  radio  Shd  x^adar.  Itezyis  were  divided  between  theoretical  and  perfortriance 
types.  Examples  d teebreticai  items  are  "principles  of  regenerative  and  degenerative 
feed-back**  and  *til)srhdterlstics  of  aeries -resonant  circuits.*  Typical  performance  Items 
arc  "perform  familiarization  exercises  on  oscilloscope **  and  “plot  selectivity  curves  of 
receivers,"  Ratings  were  on  a five  point  scale  extending  from  “absolutely  essential  to 
job  proficiency"  to  "of  no  value  to  job  proficiency."  Electronics  technicians  were  the 
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oaly  |s®r8osi®  from  wltom  'Oiis  icformstion  w»is  received. 

Cae  of  the  least  coRvestloaai  methcds  used  a card  sort.  It  introduced  in  an 
attern|>t  to  get  the  men  to  describe  their  Job  in  a cormncin;  or  uniform  language  by  pre- 
aenvTSSg  to  tRem  a targe  nuniber  of  Job  activi^  statements  printed,  on  cards  ana  n&ving 
them  select  those  activities  wh.tch..thev  do  as  part  of  tlieir  iob.  In  addition,  it  wits  pcs- 
ffiiMe  to  have  the.  men  rsialw  about  the  Job  BSaiemente  (such  aa  the  amoimt  of 

electronics  comprehension  which  each  activity  required)  by  sorting  them  into  atructiurcd 
categories . 

A deeV,  <o£  24?  cards  was  used.  The  face  of  each  contained  a statement  which  de- 
scribed  a certain  type  e£  beh&vtor  which  might  be  rel«va.Tst  tc  the  dut>'  ssaigrtmeais  of  the 
various  men.  Items  such  as  •sychronissc  PPI  sweeps"  “mamialiy  tune  cavity  resonator" 
and  “match  impedance  in  double  slug  trsnsmissioTi  line"  are  typical.  All  electrcriicp 
technicians  soi*ted  the  items  according  to  relevance  to  their  job,  frequency  of  perform- 
ance, where  they  first  learned  to  perform  the  activity,  and  the  ameuata  of  clrctronics 
comprehenaton  and  skill  required  in  pterforming  the  activity.  Rated  soBartnen  a®d*the 
lead  radarman,  lead  radiemaa,  and  the  fire  ceaitrol  techaiciaa  convicted  ail  of  the  above 
exxept  the  comprehension  and  skiil  sorts , 

Each  observer  kept  a written  narrative  account  of  each  trip.  This  account  wa* 
called  the  observer's  log,  or  sometimes  the  log  method,  allhough  it  was  not  primarily  a 
research  devlcd.  The  coli^nt  of  each  log  wa*  determined  by  the  particular  observer.  He 
wraa  ettcodraged,  however,  to  bnclode  all  infomation  of  value  which  was  not  collected  In 
some  other  manner. 

The  log  served  as  a means  of  communication  between  the  observers  in  the  field 
and  the  other  members  of  the  research  group.  It  was  also  a handy  irtstrumeat  for  re- 
cording research  Ideas  that  cccured  to  the  investigator  in  the  course  of  his  obaerva- 
tlons.  Frequently  a casual  Conversation  would  be  quite  pertinent  to  the  research,  and  in 
theid^^hied  ^he  bha  would  iatiude  hn  tthttract  of  the  conversation  in  his  log. 

Repair  records  maintained  by  electronics  technicians  provide  a potential  source 
of  information  on  such  problems  as  causes  of  electronic  failure,  types  of  maintenance 
activities,  equipment  on  which  time  is  spent,  and  characteristic  problems  of  given  equip- 
ment. To  collect  this  information  for  later  analysis  a record  summary  form  was  used. 
Repair  data  were  copied  from  the  shipboard  records,  (usually  equipment  logs  and  repair 
record  notebooka)  to  cover  the  period  of  the  last  complete  calendar  mon^.  Information 
recorded  included  the  date  of  the  repair,  equipment  unit  involved,  nature  of  trouble  and 
location,  work  done,  parts  used,  operator  on  equipment  when  it  broke  down,  and  the  rate 
of  the  person  who  made  the  repair. 

To  provide  a more  complete  record  for  the  period  that  obseriwrs  were  aboard, 
the  electronics  technicians  were  asked  to  complete  a repair  record  on  their  next  cor- 
rective maintenance  operation.  This  record  was  more  detaileii  than  those  usually  kept 
by  the  men.  In  addition  to  the  conventional  items  on  equipment,  nature  of  trouble,  and 
parts  used  (as  described  above)  this  form  requested  Information  on  all  circuits  tested, 
all  parts  tested,  front  panel  indications  of  trouble,  test  equipment,  tools  and  instruction 
books  used,  etc.,  plus  a step  by  step  description  of  work  done  from  the  time  of  notifica- 
tion of  trouble  until  the  repair  was  completed.  The  step  by  step  description  included  out- 
side interferences  encountered,  non-maintenance  activities  performed,  and  aid  or  super- 
vision received,  as  well  as  tests  and  adjustments  made. 

Since  there  is  some  information  which  is  difficult  to  collect  with  written  and 
more  formalized  methods,  an  interview  was  scheduled  with  each  electronics  technician 
end  the  CIC  and  electronics  repair  officers . In  most  instances  the  answers  to  specific 
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q’uestioiis  were  obtained,  slthougb  the  jierssm  interviewed  wa*  encoureged  to  esqsmnd  on 
any  |>oint  he  desired,  in  the  case  the  eiectroTiics  tcchsaciaas  the  ^questions  centered 
arosmd  each  tnatters  as  sriptivatlon,  future  plans,  and  job  |»referem:e3«  OSicers  were 
asked  fh*tr  prirai.ry  ivnd  se^ottuary  nuctes,  the  degrees  to  which  Idtey  ei^ouraged 

tty  their  soct;*  ^ the  ,iaterrela±ion*d»lpB' ■•‘^ffi-ce-r-reopcnstbill-’ 

ttcs,'  «nd  slrmlar  <|\2e9ticnB,  About  one  ^'haM  hour  was  devoted  to^^tiie  ^disenss'km'of  'tisese 
matters,  and  a recording  cm  rriiagnetic  tape  was  kept  of  the  conversation. 

Ail  electronics  tecbnicieiw  were  Rdmf.”‘-Stsred  ihi  Job  liuesUoimatire,  the  general 
questionnaire,  the  trainiog  questlcrrjaire,  and  the  card  sort.  In  addition  they  filled  ou£ 
repair  records,  were  Interviewed,  and  provided  critical  incidents.  The  lead  ET  also 
eomplsted  the  check  list  and  the  ability  requirements  seals.  AP  senArmen  cosnpjcted  the 
job  qi^estiosaasirc.  Plated  sonarmen  performed  the  card  sort,  aad  the  lead  SC  filled  out 
the  check  Mat,  Aii  radarmen  and  radiot?>ern  completed  the  job  questioimBlre,  and  tibe 
leads  of  each  group  prepared  the  check  list  and  perfoj;.n.ied  the  card  sort.  The  fire  con-' 
troJ  technician  completed  the  Job  questionnaire,  checklist,  and  card  sort. 

Officers  who  h&d  electrecics  sipecialista  dixectiy  tmder  their  supervision  com- 
pleted  the  general  questionnaire  and  the  abUity  requirermnis  scale.  This  group  included 
the  CIC,  ASW,  communications,  gunnery,  and  electronics  repair  officers  (the  latter  also 
completing  a job  questionnaire  and  check  Ust,  In  addition,  the  CIC  and  electronics  offi- 
cer were  interviewed.  The  operations,  engineering,  and  executive  officers  (and  In  most 
instances  the  commanding  officer)  were  invited  to  complete  the  general  questionnaire. 

When  the  observer  was  not  administering  the  card  sort,  interviewing,  or  helping 
men  complete  the  questionnaires,  he  spent  his  time  taking  man-,  pXsce-,  and  trouble- 
oriented  diaries.  When  word  of  equipment  failure  was  received,  he  would  leave  whatever 
he  was  doing  to  go  to  observe  the  trouble  shooting.  He  carried  a check  list  with  him  at 
aU  times,  making  entries  upon  It  as  he  observed  men  performing  the  activities  listed. 
Every  few  houre  he  would  make  the  necessary  additions  to  his  personal  log  to  keep  that 
record  complete. 


TREATMENT  OF  DATA 


The  analysis  of  the  data  was  aitnpJe  and  straightforward.  Its  form  was  largely 
determined  by  the  limitations  of  the  sample  and  the  objectives  of  the  research.  The  d&ta 
were  treated  in  a number  cf  different  ways.  The  most  often  used  form  of  analysis  was 
the  simple  frequency  count.  This  operation  was  employed  extensively  to  answer  such 
questions  as,  “How  many  men  of  a given  rate  lubricate  PPl  slip  rings  as  a part  of  their 
normal  job?"  “’How  many  men  answered  ‘yes*  to  a given  question?**  and  "On  how  many 
ships  was  such  and  such  the  case?"  Percentages  were  computed  in  many  cases  to  per- 
mit a comparison  of  the  agreements  and  disagreements  between  the  various  rates  and 
ratings.  The  use  of  percentages  was  required  because  of  the  highly  variable  number  of 
cases  in  each  of  the  sub-saxnplss.  Ranking  procedures  were  introduced  in  order  to  in- 
dicate the  most  frequently  used  tools,  publications,  and  job  activities.  Individual  rankings 
were  In  many  instances  converted  into  rankings  representative  of  the  group.  The  desire 
to  present  the  information  as  characteristic  o£  a given  group  led  to  the  computation  of 
conventional  expressiOBS  of  central  tendency  and  related  measures  of  dispersion.  The 
basic  treatment  of  the  critical  incident  data  consisted  of  a logical  organization  of  the  key 
behaviors  in  each  incident  into  mutually  excl'isive  categories. 
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EVAI,TjATION  OF  THE  MSTffC8^. 


Is  ii<rM  «it  ia  tbe  .flees  .it  ia  £^.aoe%lM>as 

regard  to  the  sxfcsat  to  which  e».cJi  coatribctcd  the  taforifpa%^  wMch  it  '«r»s  deeigoed, 

^ ' - TL. ' -Sif *_.r.  ._#*  .B-iL  _ _ ..  9 ■*  .A  j»<*  _ ._  ^ . _ _ , . ' ' ^ '■inL.  ’■  .ki  ■«  . >•%  ..  ij*„.  tL  * 


A rd«gk'  divisiij®  of  the  Into  three  ciistfesed 


ea.«3  fatiota  '-com- 


trvbui’mg  U>  ihE'  evaiu-a.i.ivia  'S'h.tS-h 


Tip  to  ts.ll  ex|5>eet«tit,>ri.«,  (2)  those  'A’Mch  were  tsmited  fe«t  aot'i 
which  failed  in  sosne  way  to  .meet  desired  .reqairestscnfes.* 


% *sad  (3).  teclmiqiae'it 


Thj&  c^td  sort  sn-d  sll  of -Sie  questio?fm5re pro%^e4  v’?»ry 

their  purpose.  The  card  sort  was  well  accepted  fey  the  men  with  the  reaitSi  that  their 
sii'ovivation  was  easily  sustained.  The  itiformation  gais.ed  otJccUve  asid  capafele  d£ 
majiy  possible  forms  of  asslysis.  Its  sfeief  IhTjifcstion  tw.rned  out  to  fee  tfee  anrsotaast  of  ob- 
server time  that  was  required  fcr  it®  sdmiaistraiion,  Twe  sty  to  twenty-ft’^  hours  were 
required  per  ship  lor  this  purpose,  AD.  <a  the  queatiojsRj^ires  shared  the  wavftm*g®  ta 
ease  of  administration  and  made  possible  Die  collection  s?/,  a roaximurtt  amotESt  is^or- 
imDort  without  interfering  with  shipfeosrd  activities.  Because  most  of  t3s.e  q«e*ttote»  tt’er« 
worded  to  yield  objective  answersj,  analysis  of  the  dafa  received  waa  fac'Ditated,  Tl«e 
oaiy  difficulty  encountered  was  the  tendency  of  Die  JiSfen  to  confer  about  their  saawcirh, 
though  they  had  been  requested  to  work  tndependenDy«TQnestiomialres,  of  cOarsC,  Cmdd 
readily  be  tised  without  requiring  the  observers  to  travel  aboard  the  ships « 


The  period  cf  er  five  dsi^ys  fer  f»b«ervatiwi  on  each  *hin  b;nnoacd  limiiaDons 

oti  the  •oppo.^t unity  to  observe"  for  the  methods  requiring  direct  obwervatioa  i^uch  ss  the 
diary  and  the  observer  completed  check  list.  There  v,»ere  situafcios*  where  no  iroubhs» 
oriented  diaries  were  taken  during  an  entire  week  of  operations  because  ns>  trouble  de- 
veloped. In  this  situation,  the  ofesirihErs"'  cMck  list  is  very  sparsely  dheckad.’*''T!sera  it 
little  that  can  be  done  to  overconve  these  obstacles  except  to  extend  the  period  o£  ob- 
servation, It  i»  important,  therefore,  that  the  sample  of  situations  observed  satisfy  the 
criterion  of  representativeness.  The  Checklist  was  not  sublect  to  these  limitaDoas  when 
filled  out  fey  supf;rvisor8. 


Perhaps  as  much  as  any  other  method,  the  critical  incident  interview  demon- 
strated the  need  tor  careful  pretesting  of  questions  and  training  of  interviewers.  In  the 
case  of  one  of  the  questions  it  was  quite  difficult  to  elicit  descriptions  of  particular  situ- 
ations. Difficulty  wa^  also  eucouatcred  in  obtaining  as  many  as  four  different  incidents 
from  D*e  same  man  witl5  reference  to  the  same  question.  In  addition,  care  had  to  be 
taken  to  keep  from  getting  superficial  or  "generaliy-ed"  incidenis.  Wfeon  these  problems 
were  carefully  attended  to,  the  critical  Incident  interview  proved  a fruitful  source  of  la- 
lormbtlon. 

The  general  interviev/  did  not  piove  maximally  successful  for  two  reasons , In 
the  first  place  it  was  greatly  influenced  l>y  limitaDoa*  of  space  in  which  to  carry  it  out. 
Second,  and  more  important,  waa  the  lack  of  sulficicot  structuring  of  tfe  interview. 
Early  in  the  study  an  effort  was  made  to  accent  informality,  opon-ended  questions^  and  a 
non-directive  approach.  This  did  not  prove  satisfactory,  and  it  w&s  later  concluded  that 
to  be  maximally  usefxil  under  the  circumstances,  the  interview  gfcouJd  ixs  highly  sir vse- 
lared. 

The  record  summary  wes  iim^'bsd  fey  the  lack  of  umfonrii  data.  There  waa'  so 
much  'variation  in  record  keeping  practices  from  ship  to  ship  that  the  date  obtained  froiyi 
that  source  was  highly  variable. 


In  svmim«?y;  earr-  sort.  ,|*&w  fe?4ersi.  qu-jistlrsnaatre ,■  a-sd  train-* 

i&g  <ij'iseSfcioT»n&ir«  v?ere  fotmd  te  be  the  ttiost  methods  iirthe  isa»R«  rsf 

\xtey  designed.  Tlwf  4ia:ry,  efefeck  Keyt,  critical  incident,  ‘ 

!te§dMe>  ,:»|id',  tib0.ity  Tea'uiye«>tff--n*» 

iimttwtiflsifi,.;..  Tfe  isde^iew  .need,  ior  -fead  ieftcoT'd  ■vfetjr'K-'y  ws^-i; 

'KimteS.  liseluiccss-. 


EVALUATION  OF  THE  MULTI --METHOD  A Pi'TvOACH 


At  the  coTtclu5t«ws  c£  siich  en  observational  c5'ort  as  tl^.oae  'aadetiakar,  it 
shoaid  be  possible  to  "befeiti^te  “^ks  pro^am  Sts  a wbe3.c-,  'O^e  ibgit»snai.iely  ask  tisj 

‘"’Did  the  mxil’^-WSelhiSd  h5?i|ir‘s^  come  up  ie  e»iie€t*.tfati3S»>;feed  iS  aot,  biow  -did  it 

faHtthom*  ' ' "' 

Perhaps  the  most  impertshi  objection  to  this  ;2ippeciac3|'ts  ib*it  it  over lo«.ded 
Observers.  With  twelve  different  methods  tp  fep  observer  had  feo  iit-^ 

tie  time  to  make  the  onstruebired  observatii^jS  .“wklch  ©rifla^lly  b%t«ndi?,4  to 

ft  central  position  tn  the  study,  li  some  fot'tA  ^ full,  deals  djress-jreheaTsai  h©.d  bisen  ii«e'4 
this  diificvlty  would  h»ve  bacomt  apparent  c«pl4,lsaee  bifes.  corrected  ia  advabce. 

A second  difficulty  that  arose  from  the  vifee  df  more  Ihaa  add  method 
from  the  fatt  ttat  the  various  rficihods  -^ere  ei  dlSerdnt  in  ■^terir  admihistratioR  asd 
types  of  response.  As  ft  reaullf  the  indivldna?  16  **pXiAu*"  i-eitiiii  mesb,- 

ods  oa  short  crutftes.  Many  fseters  went  into  the  declsloas  th«.i  were  made  by  the  ob- 
servers, but  the  overall  effect  was  that  the  structured  jneiisodft  such  ag>  the  qaestion-- 
nftires  were  completed  e»rly  In  the  trip  whUd'^fe/vi»''I^Jtfe^rbd 
-sad  to  bitid  of  the  trip.  DH^ih ''FeMarttbld''^'^0%iMy  %6^d  'pd 

Despite  the  diJKioxlttes  Indicftted  above  the  multi'-metkod  ft|iproftch  wsjs  generally 
Sfttiftfftctory.  It  provided  & mftny-Bided  picture  of  an  ertrca?ely  tater'depeadent  and  com^ 
plex  domain,  the  concurrent  use  of  the  several  techniques  permitted  the  Comparisoft  and 
tentative  evaluation  of  a number  of  potewEtal 'research  methods,  Thift  #aiR  lar^siy  due  to 
the  fact  that  the  methodj  were  applied  to  a single  captive  sample.  Most  importent  csf  ail, 
the  multi-methc  '.  approach  yielded  a set  zsf  descriptions  which  were  broad  hn  scope  yet 
rich  in  detail. 


